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Abstract 

In this paper we prove Chaitin's "heuristic principle" , the theorems of a finitely- 
specified theory cannot be significantly more complex than the theory itself, for an appro- 
priate measure of complexity. We show that the measure is invariant under the change 
of the Godel numbering. For this measure, the theorems of a finitely-specified, sound, 
consistent theory strong enough to formalize arithmetic which is arithmetically sound 
(like Zermelo-Fraenkel set theory with choice or Peano Arithmetic) have bounded com- 
plexity, hence every sentence of the theory which is significantly more complex than the 
theory is unprovable. Previous results showing that incompleteness is not accidental, but 
ubiquitous are here reinforced in probabilistic terms: the probability that a true sentence 
of length n is provable in the theory tends to zero when n tends to infinity, while the 
probability that a sentence of length n is true is strictly positive. 



1 Introduction 

Godel's Incompleteness Theorem states that every finitely-specified, sound, theory which is 
strong enough to include arithmetic cannot be both consistent and complete. Godel's original 
proof as well as most subsequent proofs are based on the following idea: a theory which is 
finitely-specified, sound, consistent and strong enough can express sentences about provability 
within the theory, which, themselves, are not provable by the theory, but can be shown to be 
true using a proof by contradiction. A true and unprovable sentence is called independent. 
This type of proof of incompleteness does not answer the questions of whether independence 
is a widespread phenomenon nor which kinds of sentences can be expected to be independent. 

Chaitin ^1] presented a complexity-theoretic proof of Godel's Incompleteness Theorem which 
shows that high complexity is a reason of the unprovability of infinitely many (true) sen- 
tences. This result suggested to him the following "heuristic principle" , a kind of information- 
preservation principle: the theorems of a finitely specified theory cannot be significantly more 



complex than the theory itself. This approach would address the second of the questions above, 
that is, highly complex sentences are independent, and, as a consequence, would indicate that 
independence is pervasive. A formal confirmation of the pervasiveness of independence has 
been obtained in via a topological analysis; a quantitative result is still missing. 

In this paper we prove that a formal version of the "heuristic principle" is indeed correct for 
an appropriate measure of complexity; the measure is invariant under the change of the Godel 
numbering. For this measure, 6, the theorems of a finitely-specified, sound, consistent theory 
which is strong enough to include arithmetic have bounded complexity, hence every sentence 
of the theory which is significantly more complex than the theory is unprovable. Previous 
results showing that incompleteness is not accidental, but ubiquitous are here reinforced in 
probabilistic terms: the probability that a true sentence of length n is provable in the theory 
tends to zero when n tends to infinity, while the probability that a sentence of length n is 
true is strictly positive. 

The paper is organized as follows. In Sections 2 and 3 we present the background, the notation 
and main results needed for our proofs. In Section 4 we discuss some general complexity- 
theoretic results which will be used to prove the main result (Theorem 14. 6() . In Section 5 
we prove that incompleteness is widespread in probabilistic terms. In Section 6 we use the 
new complexity measure to prove Chaitin's information-theoretic incompleteness result for 
the Omega Number. We finish with a few general comments in Section 7. The bibliography 
includes a selection of relevant papers and books, but is by no means complete. 

2 Background 

Godel's Incompleteness Theorem, announced on 7 October 1930 in Konigsberg at the First 
International Conference on the Philosophy of Mathematics^ is a landmark of the twentieth 
century mathematics (see (SSHSHI^ for the original paper, [SIIIHIEIIHS] for other proofs and 
more related mathematical facts, PHI EHl 11011111 ED 1^ EH 0113123111 for more mathematical, 
historical and philosophical details). It says that in a finitely-specified, sound, consistent 
theory strong enough to formalize arithmetic, there are true, but unprovable sentences; so 
such a theory is incomplete. A true and unprovable sentence is called independent. The 
first condition states that axioms can be algorithmically listed; consistency means free of 
contradictions; soundness means that any proved sentence is true. 

According to Hintikka (|3S1) P- 4), with the exception of von Neumann, who immediately 
grasped Godel's line of thought and its importance, incompleteness passed un-noticed in 
Konigsberg: even the speaker who summarized the discussion omitted Godel's result. In 
spite of being praised, discussed, used or abused by many authors, the Incompleteness The- 
orem seems, even after so many years since its discovery, stranger than most mathematical 
theorems.^ For example, according to Solovay fl37j. p. 399): "The feeling was that Godel's 
theorem was of interest only to logicians"; in Smoryiiski's words, (|SZ1; P- 399), "It is fash- 
ionable to deride Godel's theorem as artificial, as dependent on a linguistic trick." 

In 1974 Chaitin ^1] presented a complexity-theoretic proof of Godel's Incompleteness Theo- 
rem which shows that high complexity is a reason of the unprovability of infinitely many (true) 
sentences. This complexity-theoretic approach was discussed by Chaitin [TBllTTllTHlOniOTll^ 

^Hilbert, von Neumann, Carnap, Heyting presented reports; the conference was part of the German Math- 
ematical Congress. 

■^This is quite impressive, as mathematics abounds with baffling results. 
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and various authors including Davis [2^, Tymoczko ISO], Boolos and Jeffrey [S], pp. 288-291, 
Svozil HO], Li and Vitanyi jUj, Barrow [HEl, Calude Calude and Salomaa HU, Casti 

jl2j . Delahaye [^; it was critized by van Lambalgen [Snj, Fallis Raatikainen Hin- 
tikka ES]. 

Chaitin's proof in is based on program-size complexity (Chaitin complexity) H: the 
complexity H{s) of a binary string s is the size, in bits, of the shortest program for a universal 
self-delimiting Turing machine to calculate s. The complexity H{s) is unbounded. The proof 
shows that for every finitely-specified, sound, consistent theory strong enough to formalize 
arithmetic, there exists a positive constant M such that no sentence of the form "H{x) > m" 
is provable in the theory unless m is less than M . There are infinitely many true sentences 
of the form "//(x) > m" with m > M, and each of them is unprovable in the theory. 

The high //-complexity of the sentences (x) > m" with m > M is a source of their unprov- 
ability.^ Is every true sentence s with H{s) > M unprovable by the theory? Unfortunately, 
the answer is negative because only finitely many sentences s have complexity H{s) < M in 
contrast with the fact that the set of all theorems of the theory is infinite. For example, ZFC 
(Zermelo-Praenkel set theory with choice) or Peano Arithmetic trivially prove all sentences 
of the form "n + 1 = 1 + n" . The //-complexity of the sentences "n + 1 = 1 + n" grows 
unbounded with n. This fact, noticed and discussed by Chaitin in Section 6 of [HI (reprinted 
in [2^ pp. 55-81) as well as by Svozil [321; PP- 123-125, is essential for the critique in [301115] 
(cited in ■35i); the works [121[^[ini seem to be unknown to the authors of [^011131153] . 

Chaitin's proof based on H cannot be directly extended to all unprovable sentences, hence 
the problem of whether complexity is a source of incompleteness remained open. In this note 
we prove that the "heuristic principle" proposed by Chaitin in [21j . p. 69, namely that the 
theorems of a finitely- specified theory cannot he significantly more complex than the theory 
itself ^ is correct if we measure the complexity of a string by the difference between the 
program-size complexity and the length of the string, our 5-complexity (Theorem 14. 6j) . The 
//-complexity of the sentences "n -|- 1 = 1 -|- n" grows unbounded with n, but the "intuitive 
complexity" of the sentences "n -|- 1 = 1 -|- n" remains bounded; this intuition is confirmed by 
^-complexity. Note that a sentence with a large (5-complexity has also a large //-complexity, 
but the converse is not true. There are only finitely many strings with bounded //-complexity, 
but infinitely many strings with bounded 5-complexity. 

As a consequence of Theorem 14.61 we prove that the incompleteness phenomenon is more 
widespread than previously shown in [221 1^ 1201 1^ and by the topological analysis of 
[0]: the probability that a true sentence of length n is provable in the theory tends to zero 
when n tends to infinity, while the probability that a sentence of length n is true is strictly 
positive. 

'^Fallis l^-iO) . p. 264, argued that Godel's true but unprovable sentence G is likely to have excessive H- 
complexity. Similarly, if the theory is capable of expressing its own consistency, then the corresponding 
sentence is likely to have excessive //-complexity. It would be interesting to have a mathematical confirmation 
of these facts. 

^An "approximation" of this principle supported by Chaitin's proof is that "one cannot prove, from a set 
of axioms, a theorem that is of greater //-complexity than the axioms and know that one has done it"; see 
|21) . p. 69; see also Theorem 14. 71 in Section 4. 
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3 Prerequisites 



We follow the notation in 0. By IN = {0, 1, 2, . . .} we denote the set of non-negative integers. 
Further on, logQ denotes the base Q > 2 logarithm and logn = [log2(n + 1)J; [aj is the 
"floor" of the real a and \a\ is the "ceiling" of a. The cardinality of the set A is denoted by 
card (A). An alphabet with Q elements will be denoted by Xq; by Xq we denote the set of 
finite strings (words) on Xq, including the empty string A. The length of the string w € Xq 
is denoted by \w\q. 

For Q = 2 we use the special set B = {0, 1} instead of X2. We consider the following bijection 
between non-negative integers and strings on B: A, 1 1-^ 0,2 i-^^ 1,3 1— > 00,4 1— > 
01,5 I— > 10,6 ^ 11,... The image of n, denoted bin{n), is the binary representation of 
the number n + 1 without the leading 1. Its length is \bin{n)\2 = logn. In general we 
denote by stringQ(n) the nth string on Xq according to the quasi- lexicographical order. In 
particular, bin{n) = string2{n). In this way we get a bijective function stringQ : INf — > Xq] 
\stringQ{n)\Q = [logQ(n(Q - 1) + 1)J. 

We assume that the reader is familiar with Turing machines processing strings, computability 
and program-size complexity (see, for example, 01 El 1201 ) • "^^^ program set (domain) of 
the Turing machine T is the set PROGt = {x € Xq : T halts on x}; when T halts on x, 
r(x) is the result of the computation of T on x. A partial function (/? from strings to strings 
is called partial computable (abbreviated p.c.) if there is a Turing machine T such that: a) 
PROGt = dom(99), and b) T(x) = 9?(x), for each x G PROGt- A computable function 
is a p.c. function ip with dom((/j) = Xq. A set of strings is computable if its characteristic 
function is computable. A set of strings is computably enumerable (abbreviated c.e.) if it is 
the program set of a Turing machine. 

A self- delimiting Turing machine is a Turing machine T such that its program set is a prefix- 
free set of strings. Recall that a prefix-free set of strings S is a set such that no string in S 
is a proper extension of any other string in S. In what follows the term machine will refer to 
either a p.c. function with prefix-free domain or a self-delimiting Turing machine. 

Each prefix-free set S C Xq satisfies Kraft's inequality: '''i ' Q * — 1; where = 

card {x G S : \x\q = i}. A stronger result, the Kraft-Chaitin Theorem (see (HI, p. 53), 
is essential in algorithmic information theory: Let ni,n2,... be a computable sequence of 
non-negative integers such that 

00 

Y^Q'^^<1. (1) 

j=i 

Then, we can effectively construct a prefix- free sequence of strings wi,W2, ■ ■ ■ such that for 
each i > 1, \wi\Q = Ui. 

The program-size complexity of the string x G Xq (relative to T) is Hq^t{x) = min {\y\Q : y G 
Xq, T{y) = x}, where min0 = 00. The Invariance Theorem states that we can effectively 
construct a machine U = Uq (called universal) such that for every machine T there exists a 
constant e > such that for all x G Xq, Hq^ij{x) < Hq^t{x) + £• In what follows we will fix 
U and put Hq = Hq^u; in particular, H2 denotes the program-size complexity induced by a 
universal (binary) machine. If x is in Xq, then x* = min{n G Xq : Uq{u) = x}, where the 
minimum is taken according to the quasi-lexicographical order; it is seen that Hq{x) = \x*\q. 
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4 Complexity and Incompleteness 



In this section we introduce the (^-measure and then prove for it Chaitin's "heuristic principle" : 
the theorems of a finitely- specified theory cannot be significantly more complex than the theory 
itself. 

First we introduce the (5-measure. Recall that Uq is a fixed universal machine on Xq and 
Hq = Hq^Uq- In what follows we will work with the function 5q{x) = Hq{x) — \x\q (note 
that —6q is a "deficiency of randomness" function in the sense of |6j, Definition 5.21, p. 113). 
The (5-complexity is "close", but not equal, to the conditional i/g-complexity, of a string 
given its length. 

The complexity measures Hq and 5q have similarities as 5q is defined from Hq by means of 
some simple computable functions; for example, they are both uncomputable. But Hq and 
5q differ in an essential way: given a positive A'^, the set {x € Xq : Hq{x) < is finite 
while, by Corrolarv 14.31 the set {x G Xq : 5q{x) < N} is infinite. Note that the conditional 
iifQ- complexity does not have this property. A sentence with a large (Jg-complexity has also 
a large ffg-complexity, but the converse is not true. For example, the -ffg-complexity of a 
(true) sentence of the form "1 + n = n + 1" is about [logg nj plus a constant, a function 
which tends to infinity as n — > oo; however, their Jg-complexity is bounded. 

In view of Theorem 5.4 in 0, p. 102, there exists a constant c > such that: 

max 5q{x) > HQ{stringQ{N)) - c, (2) 

\x\q=N 

so there are strings of arbitrarily large (Jg-complexity. 

The following result is taken from [S] (Theorem 5.31, p. 117). 

Theorem 4.1 For every t > 0, the set Cq^i = {x G Xq : Sq{x) > —t} is immune, that is, 
the set is infinite and contains no infinite c.e. subset. 

Corollary 4.2 For every t >0, the set Complex = {x £ Xq : 5q{x) > t} is immune. 

Proof. As Complex C Cg^t and every infinite subset of an immune set is immune itself, 
we only need to show that Complex g^^ is infinite. To this aim we use formula Q and the 
fact that the function HQ{stringQ{N)) is unbounded. □ 



Corollary 4.3 For every t > 0, the set {x G Xq : 6q{x) < t} is infinite. 

Proof. The set in the statement is not even c.e. because, by Corollarv 14.21 its complement 
is immune. □ 

The above result suggests that any "reasonable" theory cannot include more than finitely 
many theorems with high (5-complexity. And, indeed, a simple analysis confirms this fact. 
A formal language used by a theory capable of speaking about natural numbers includes 
variables (a fixed variable x and the sign ' may be used to generate all variables, x,x',", 
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etc.), the constant 0, function symbols for successor, addition and multiplication, 
the sign for equality, =, logical connectives,-!, A, V, =>, quantifiers, V, 3, and parentheses, (,). 
They form an alphabet Xi^.^ The formal language consists of well- formed formulae which 
respect strict syntactical rules; for example, each left parenthesis has to be matched with 
exactly one right parenthesis. Theorems are then defined by specifying the axioms and the 
inference rules. For instance, the system Q introduced by R. M. Robinson (see, for example, 
j29j ) contains the logical axioms (propositional, substitution, V-distribution, equality axioms) 
and the following seven axioms: Ql: {s{x) = s{x')) =^ (x = x'), Q2: -i(0 = s(x)), Q3: 
(^(x = 0)) 3x'{x = s{x')), Q4: x + = x, Q5: x + s(x') = s{x + x'), Q6: x • = 0, 
Q7: X ■ s(x') = (x • x') + x, and the inference rules of modus ponens and generalisation. 
A proof in the system Q is a sequence of well-formed formulae such that each formula is 
either an axiom, or is derived from two earlier formulae in the sequence by an inference 
rule. Theorems are well- formed formulae which have proofs in Q. As theorems are special 
well-formed formulae, it is clear that each theorem x in the system Q has rather small Hi^- 
complexity, more precisely, Hi^{x) is not larger than its length plus a fixed constant. Such 
a remark suggests that Chaitin's "heuristic principle" may be true for 5i^. However, this 
property could be a consequence of some particular way of writing/coding the theorems! To 
be able to measure somehow the "intrinsic" complexity of a theorem we need to prove that 
the property is invariant with respect to a system of acceptable names, in our case, Gddel 
numherings. 

To make the discussion precise, let us fix a formal language L C Xq. A Gddel numbering for 
L is a computable, one-to-one function g : L ^ B* , i.e. a system of unique binary names for 
the well-formed formulae of L. For example, a Godel numbering for the well-formed formulae 
of the system Q can be obtained by coding the elements of the alphabet Xi^ with the first 
15 binary strings of length four, and then extend this coding according to the syntax of the 
language. Various other possibilities can be imagined; see for example, [3l l29]. 

As the set of theorems is a c.e. subset of the set of well-formed formulae, we will work only 
with computable, one-to-one functions g : T ^ B* defined on the set of theorems. 

The 6-complexity of a theorem u & T induced by the Gddel numbering g is defined by: 

6g{u) = H2ig{u))-\log^Q]-\u\Q. (3) 

The formula for Sf is essentially the formula defining 5q relativized to the Godel numbering 
g: the factor [log2 Q~\ has the role of "adjusting" the sizes of the alphabets Xq and B. 

The first result confirms the intuition: we prove that 5g is, up to an additive constant, equal 
to [logaQl • 6q. 

Theorem 4.4 Let T C Xq be c.e. and g : T ^ B* be a Godel numbering. Then, there 
effectively exists a constant c (depending upon Uq,U2, and g) such that for all u ^ T we 
have: 

\5g{u) - \\og2Q^ ■ 5q{u)\ < c. (4) 

Proof. First we prove the existence of a constant ci such that 

(5g(n) < [logaQl •5q(u) + ci. (5) 
'''Of course, we can work with smaller or larger alphabets, depending on specific needs. 
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For each string w € PROGjjq we define = [log2 Q] • |w|q, and we note that 

w€PROGuq wdPROGuQ w€PROGuq 

because PROGjjq is prefix-free. Using now the Kraft-Chaitin Theorem, we can effectively 
construct, for every w G PROGuq a binary string Sw such that \sw\2 = and the set 
{sw '■ w £ PROGjjq} is c.e. and prefix-free. This allows us to construct the machine C 
defined by 

C{s^) = 9{Uq{w)), for w G PROGuq- 
As G{syj*) = g{UQ{w*)) = g{w) we have 

Hc{g{w)) < |s^„.|2 = flogaQl • \w*\q = [logaQ] • Hq{w). 

Applying the Invariance Theorem we get a constant ci > such that 

5g{w) = H2{g{w)) - [logaQl • \w\q 

< [log2 Q] ■ {Hq{w) - \w\q) + Cl 
= [logaQl • 5q{w) +ci, 

which proves ©. 

Secondly we prove the existence of a constant C2 such that 

{log^Q-] ■ 5q{u) < 5g{u) + C2. (6) 

For each w G PROGu2 such that \w\2 > log2(5, we put rriw = \\w\2 • logg 2] > 1, and we 
note that 



W)ePi?,OG(72,|to|2>log2 Q Wl^PROGu2,\w\2>log2 Q 

hence, in view of the Kraft-Chaitin Theorem, we can effectively construct, for every w G 
PROGU2 with \w\2 > log2(55 a string tw G Xq of length |tu,|Q = such that the set 
{tw : ?/; G PROGu^} is c.e. and prefix-free. In this way we construct the machine D defined 
by D{tw) = n if U2{w) = g{u). This construction is well-defined because 5 is a Godel 
numbering. It is seen that if 1/2(10) = u and \w\2 > log2(5, then Hd{u) < \\w\2 • logQ2], so 
applying the Invariance Theorem we get a constant d such that 

riog2 Ql • Hq{u) < riog2 Q] ■HD{u)+d< H2{g{u)) + d, 

hence there is a constant C2 such that © becomes true. We have used the fact that [log2 Q] • 
\m ■ logg 2] < m, for all integers m > 0. 

Finally, © follows from © and ©. □ 



As a consequence, asymptotically, the (5-measure is independent of the Godel numbering. 

Corollary 4.5 Let T C Xq be c.e. and g,g' : T ^ B* be two Godel numberings. Then, 

there effectively exists a constant c (depending upon U2,g and g' ) such that for all u we 
have: 

\5g{u)-5g,{u)\<c. (7) 
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Proof. The relation ((Tj) follows from Theorem 14.41 However, it is instructive to give a short, 
direct proof. To this aim consider the machine C defined for w & B* hy C{w) = g{u) if 
U2{w) = g'{u). The definition is correct because PROGc C PROGu^ and g is computable 
and one-to-one. If U2{s) = g'{u), then C{s) = g{u), so by the Invariance Theorem there 
exists a constant ci such that for all u G L, 6g{u) < 5g/{u) + ci. Finally, ((TJ follows by 
symmetry. □ 



Theorem 4.6 Consider a finitely-specified, arithmetically sound (i.e. each arithmetical 
proven sentence is true), consistent theory strong enough to formalize arithmetic, and de- 
note by T its set of theorems written in the alphabet Xq. Let g be a Godel numbering for T . 
Then, there exists a constant N , which depends upon Uq, U2 and T, such that T contains no 
X with 6g{x) > N. 

Proof. Because of syntactical constraints, there exists a positive constant d such that for 
every x & T, Hq{x) < \x\q + d, i.e. Sq{x) < d (see also the discussion of the system Q 
following Corrolary 14. 3|) . Hence in view of Theorem 14.41 there is a constant N > d such that 
for every x G T, 5g{x) < N . □ 



Every sentence x in the language of T with 5g{x) > N \s unprovable in the theory; every 
such "true" sentence is thus independent of the theory. 

Do we have examples of such sentences? First, Chaitin's sentences of the form ^^H2{x) > n", 
for large n are such examples. 

Here is another way to construct true sentences of high 5-complexity. A formula (p{x) in the 
language of arithmetic is called Si if it is of the form (3y)9{x,y), where 9 contains only two 
free variables x and y. We write IN |= ip{n) to mean that <f{n) is true when n is interpreted 
as a non- negative integer. The Representation Theorem (see |48j ) states that a set i? C IN 
is c.e. iff there (effectively) exists a Si formula (p{x) such that for all n E IN we have: 
n e R ^ TN \= ip{n). 

For every a G IN, the set {n G IM : 6Q{stringQ{n)) < a} is c.e., so in view of the Representation 
Theorem there exists a Si formula 93 (depending on Uq^a) such that for every n G IN we 
have: 6Q{stringQ{n)) < a <^ JN \= ip{n). Consequently, the formula if) = -^ip represents the 
predicate ^^6Q{stringQ{n)) > a". Because of consistency and soundness, by enumerating the 
theorems in T of the form 'ilj{m) (corresponding to true formulae il)[m)) we get an enumeration 
of the set {x G T : ip{stringQ [x)) G T} C {x G T : 5q{x) > a}. 

Now let a be a non-negative integer. As {x G T : 'il){stringQ^ {x)) G T} is a c.e. subset of the 
immune set {x G Xq : 5q{x) > a}, it has to be finite, that is, there exists an M G IN such 
that for every x G T with ^{string^ {x)) G T we have: \x\q < M. We have got Chaitin's 
statement ((25' P- 69): 

Theorem 4.7 Every finitely-specified, arithmetically sound, consistent theory strong enough 
to formalize arithmetic can prove only, for finitely many of its theorems, that they have high 
5-complexity. 

The theory can formalise all sentences of the form ip{m) in a very economical way, i.e. with 
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small (5-complexity, but is incapable of proving more than finitely many instances: almost all 
true formulae of the form ^(m) are unprovable. 

Comments (a) Theorem 14.61 establishes a limit on the ^^-complexity of provable sentences 
in T; the bound depends upon the chosen Godel numbering g. In this approach it makes no 
sense to measure the power of the theory by its complexity, i.e. through the minimal N such 
that the theory proves no sentence x with dg{x) > N (see also the discussion in j39j ) . 

(b) Theorem 14.61 does not hold true for an arbitrary finitely-specified theory as there are c.e. 
sets containing strings of arbitrary large (^-complexity. 

(c) It is possible to have incomplete theories without high (5-complexity sentences; for example, 
an incomplete theory for propositional tautologies. 

5 Is Incompleteness Widespread? 

The first application complements the result of 9^ stating that the set of unprovable sentences 
is topologically large. We probabilistically show that only a few true sentences can be proven 
in a given theory, but the set of true sentences is "large" . 

We begin with the following result: 

Proposition 5.1 Let N > be a fixed integer, T C Xq be c.e. and g : T —> B* be a Godel 
numbering. Then, 

lim • card {x £ XX : \x\q = n, 6q{x) < iV} = 0. (8) 

Proof. We present here a direct proof.^ In view of Theorem 14.41 there exists a constant 
c > such that 

{x eX^: \x\q = n, 6g{x) < N} C {x £ X^ : \x\q = n, [log2 Q] ■ 6q{x) <N + c}. 

So, we only need to evaluate the limit 

lim g-" • card {x e Xp, : \x\q = n, 6q(x) < M} = 0, (9) 

?i— >oo V ^ ^ 

where M = \{N + c)/ [logs Q^^■ 

First we note that for every n we have: {x G Xq : \x\q = n, 5q{x) < M} = {x € Xq : \x\q = 
n, 3y G X^ {\y\Q <n + M, Uqiy) = x)}, so 

caid{x £ X*Q:\x\Q = n,5Q{x) <M} < cavd {y £ X^ : \y\Q < n + M,\UQ{y)\Q = n} 

< card {y£X^: \y\Q <n + M, Uqiy) halts} 



^Alternatively, one can evaluate the size of the set of strings of a given length having almost maximum 
(5Q-complexity. 
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Consequently, 

n+M 

lim • card{x G : \x\q = n,5n(x) < M} = lim V Q"" • r^, (10) 

1=1 

where = card{y G : \y\Q = i, Uq^u) halts}. Using the Stolz-Cesaro Theorem we get 

n+M m 

lim y • n = Q*^ • lim V Q"^ • n = Q^'^/{Q - 1) • lim Q"™ • = 0, (11) 

1=1 1=1 

due to Kraft's inequality ^'r^^ n-Q^^ <l. So, in view of , ©, ^ and (HTJ) we get ©• □ 



Theorem 5.2 Consider a consistent, sound, finitely- specified theory strong enough to for- 
malize arithmetic. The probability that a true sentence of length n is provable in the theory 
tends to zero when n tends to infinity, while the probability that a sentence of length n is true 
is strictly positive. 

Proof. We fix a consistent, sound, finitely-specified theory, let T be its set of theorems and 
let g he a, Godel numbering of T. For every integer n > 1, let T" = {x € T : |x|q = n}. 
By Theorem 14.61 there exists a positive integer N such that T C {x S Xq : 5g{x) < 
N}. Consequently, for every n: T" C {x G Xq : \x\q = n,5g{x) < N}, so in view of 
Proposition 15.11 the probability that a sentence of length n is provable in the theory tends 
to zero when n tends to infinity. 

Next consider the sentences hx^m = ^^Hq{x) > m", where a; is a string over the alphabet Xq. 
For every n > 1 there exists a positive integer Nm such that for every string x G Xq of length 
\x\q > Nm, hx,m is true. 

For each fixed m, \hx,m\Q = \x\q + c, so for every m > 1 and n > Nm + c we have: 

card {w G Xq : \w\q = n,w is true} • Q^" > card {x G Xq : \x\q = n — c, } • Q^"' > Q^^ , 
showing that the probability that a sentence of length n is true is strictly positive. □ 



6 Incompleteness and 

The second application is to use b to prove Chaitin's Incompleteness Theorem for Qf/ |16j 
(see also the analysis in j25( El El ) • This shows that J is a "natural" complexity. We start 
with the following preliminary result: 

Lemma 6.1 Let x\X2 ■ ■ ■ be an infinite binary sequence and let F be a strictly increasing 
function mapping positive integers to positive integers. If the set {(-F(i), xp(j)) : i > 1} is 
c.e., then there exists a constant e > (which depends upon U and the characteristic function 
of the set) such that for all k > 1 we have: 

52{xiX2---XF{k)) -k. (12) 
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Proof. To prove ()12[). for k > 1 we consider the strings: 

WlXp(^l)W2Xp(^2) ■ ■ ■WkXp(^k), (13) 

where each Wj is a string of length F{j) — F{j — 1) — 1,F(0) = 0. In this way we effec- 
tively generate all binary strings of length F{k) in which the bits on the "marked" positions 
F{l),...,F{k) are fixed. 

It is clear that X]i=i — ^i.^) ~ ^ ^^'^ the mapping {wi,W2,---,Wk) i— > wiW2 ■ ■ ■ Wk is 
bijective, hence to generate all strings of the form (|13j) we only need to generate all strings 
of length F{k) — k. Hence, we consider the enumeration of all strings of the form for 
A; = 1, 2, . . . The lengths of these strings form the sequence 

. . . , F(l), . . . , F{k),F{k), F{k), . . . 

^ V ' ^ V ' 

2^(i)-i times 2^('=)-'= times 

which is computable and satisfies the inequality © as 'Y^'^Li 2^^^'>~^ ■ 2"^^^^ = 1. Hence, by 
the Kraft-Chaitin Theorem, for every string w of length F{h) — k there effectively exists a 
string Zw having the same length as w such that the set {zyj € B* : \w\2 = F{k) — k,k > 1} 
is prefix-free. Indeed, from a string w of length F{k) — k we get a unique decomposition 
w = wi ■ • ■ Wk, and as above, so we can define C(2;^) = wiXp(^i-^'W2Xp(^2) ' " " WkXp(^ky, C is 
a machine. Clearly, 5c{wiXp{i-^W2Xp(^2) ' ' '"^kXpi^k)) ^ \zw\2 — F{k) = —k, for all k > 1. So 
by the Invariance Theorem we get the inequality (|12|1 . □ 

Consider now Chaitin's Omega Number, the halting probability of U: 0,u = 0.ujiuj2 • • • , see 
The binary sequence ujiuj2 ■ ■ ■ is (algorithmically) random. There are various ways to 
characterize randomness (see for example jl8l l5lOH]). A particular useful way is the following 
complexity-theoretic criterion due to Chaitin: there exists a positive constant fi such that for 
every n > 1, 

^2(^1^2 • • - c^n) > -fJ- (14) 
The condition ^ is equivalent to ^^^^^ 2-'^2('^i'^2-a;n) < ^f, |l2]. 

It is easy to see that the inequality ((T^ in Lemma 16.11 contradicts (fTl)) , so a sequence 
X1X2 ■ ■ ■ Xn - • • satisfying the hypothesis of Lemma l6. 1 1 cannot be random. 

Theorem 6.2 Consider a consistent, sound, finitely-specified theory strong enough to for- 
malize arithmetic. Then, we can effectively compute a constant N such that the theory cannot 
determine more than N scattered digits of Vtu = Q.ujiuj2 ■ ■ ■ 

Proof. Assume that the theory can determine infinitely many digits of 0,u = 0.ujiuj2 ■ ■ ■ 
Then, we could effectively enumerate an infinite sequence of digits of i^u, thus satisfying the 
hypothesis of Lemma Id. II which would contradict the randomness of UJ1LV2 ■ ■ ■ Q 



7 Conclusions 

There are various illuminating proofs of Godel's Incompleteness Theorem and some inter- 
esting examples of true but unprovable sentences (see for example [22^^121]). Still, the 
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phenomenon of incompleteness seems, even after almost 75 years since its discovery, strange 
and to a large extent irrelevant to 'mainstream mathematics', whatever this expression might 
mean. Something is missing from the picture. Of course, the 'grand examples' are missing; 
for example, no important open problem except Hilbert's tenth problem, see jlj, was proved 
to be unprovable. Other questions of interest include the source of incompleteness and how 
common the incompleteness phenomenon is. These two last questions have been investigated 
in this note. 

Chaitin's complexity-theoretic proof of Godel's Incompleteness Theorem ^Ij shows that high 
complexity is a sufficient reason for the unprovability of infinitely many (true) sentences. This 
approach suggested that excessive complexity may be a source of incompleteness, and, in fact, 
Chaitin (in j22[l21j ) stated this as a "heuristic principle": "the theorems of a finitely-specified 
theory cannot be significantly more complex than the theory itself" . By changing the measure 
of complexity, from program-size H{x) to 5{x) = H{x) — \x\, we have proved (Theorem 14. ()|) 
that for any finitely-specified, sound, consistent theory strong enough to formalize arithmetic 
(like Zermelo-Fraenkel set theory with choice or Peano Arithmetic) and for any Godel num- 
bering g of its well-formed formulae, we can compute a bound such that no sentence x 
with complexity 6g{x) > N can be proved in the theory; this phenomenon is independent on 
the choice of the Godel numbering. For a theory satisfying the hypotheses of Theorem 14.61 
the probability that a true sentence of length n is provable in the theory tends to zero when 
n tends to infinity, while the probability that a sentence of length n is true is strictly positive. 
This result reinforces the analysis in [2j which shows that the set of independent sentences is 
topogically large. 

According to Theorem 14.61 sentences expressed by strings with large 5g-complexity are un- 
provable. Is the converse implication true? In other words, given a theory as in the statement 
of Theorem 14.61 are there independent sentences x with low (5g-complexity? Even if such sen- 
tences do exist, in view of Theorem 15. 2| the probability that a true sentence of length n with 
5g-complexity less than or equal to N is unprovable in the theory tends to zero when n tends 
to infinity. 

Other open questions which are interesting to study include: a) the complexity of some 
concrete independent sentences, like the sentence expressing the consistency of the theory 
itself, b) the problem of finding other (more interesting?) measures of complexity satisfying 
Theorem 14. 61 c) a stronger version of Theorem l5.21 under the same conditions, the probability 
that a sentence of length n, expressible in the language of the theory, is provable in the theory 
tends to zero when n tends to infinity. 
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